slowlyfandomcom-20200214-history
User blog:Dorumin/Guide to Data Mining
You might have seen some of the stamp articles and their pristine thumbnails. Full res, transparent, and beautiful. How did I do that? Well, one hint: it didn't involve Photoshop. But I can assure you that by the end of this blog post, you will not only be able to find those images by yourself, but you'll be able to do it for any application asset in a mobile phone. We will be working with the SLOWLY app on Android and Windows in this blog, but this should work in any Apple or Linux device too. The assets Most applications come with static assets, and SLOWLY is no exception. These come bundled in the application package or APK. These are just glorified ZIP files just with a fancy extension. Let's rip one apart. Getting the APK We'll be using an app called Apk Extractor, which does exactly what you think it does. Once you have it, you can extract the APK file for any application you have installed. Search for SLOWLY and extract it somewhere you can find later. Dissecting the APK Once you have your APK file, you can move it to your desktop computer and then the fun can begin. Rename it to something like Slowly.zip so you can open it with 7zip. Once you have that down, a bunch of stuff will appear. Relax, it's gonna be fine. Navigate into res and some more folders should pop up. Now, you can go the hard way and check each one of them out, or you can just look up the image files using any search feature your file manager has. However, you may have noticed something - there are no stamps here. Not in any other folders either. That's because they aren't static assets, and are instead requested from the server on the fly. This is where proxies come in. The network A lot of the stuff you'll find in SLOWLY is dynamic and comes from the network. This is why you can't use most applications when offline (or there is a very limited amount of things you can do). That's easy to figure out, if you've ever seen a loading screen. The hard part is knowing where this stuff is coming from Meet MITMProxy MITMProxy is a wonderful Python application that allows to to easily set up a proxy on any computer or laptop easily. It also sets a very beautiful web application for you to view all traffic through your network. And because it's written in Python - it works anywhere. Once you have it installed and extracted, you should see two executables. Open up mitmweb.exe, it should boot up a cmd window and a second later a browser tab. This is where you will see all your incoming traffic, but for now, let's leave it as it is. Bring your attention back to the console window, you should see web server IP and a proxy port. Now, we need to find out our machine's local IP. Luckily for us, that's easy. Just open up the command line (you can easily Google how to do that in your operating system), and type in ipconfig on Windows or ifconfig on other machines. Excellent, we have our proxy app. Now we need to put it to good use. Connecting our device to a proxy This isn't as straightforward as it should, so bear with me, I'm know it's terrible. Feel free to re-read this as many times as you need. First, make sure your machine and phone are in the same network. All good? Alright, let's proceed. 1) Head into your WiFi settings and long press your WiFi network. 2) Select "Manage network settings" or something along the lines. 3) Head down into advanced settings and enable proxy. 4) Paste our machine IP into the "Proxy host name" textarea and our port (luckily, it's usually already set to 8080) 5) Save it. Seriously, I pressed back by accident multiple times. It's frustrating. All good? Okay, now you should not be able to connect to anything on your phone. What's up with that? HTTPs and custom certificates As the web becomes more secure and more websites transfer to HTTPs, TLS certificates are more common than ever. This is a good thing, you don't want people snooping into your web traffic, but we're going to be intercepting our own traffic for science, so we need to work around this. MITM.it is a website made to address these issues. Well, not really - the website does nothing. However, it is HTTP, so MITMProxy can intercept and modify it. Open that url on your proxied phone and you should see a pretty download site where you can get certificates for your local proxy instance. Isn't it great? Download the one for your platform. Now we need to install it. This may be a little different depending on which device you have, but we're going to be focusing on Android for now. Head into settings and search for "Credential storage". Once you're there, tap on "Install from SD card" (you can also install from local storage). Select the file you downloaded, and you're done. You may get a security notification that someone may be spying on you. Android is smart and knows something is going on - but we shouldn't worry about ourselves spying on our own traffic, right? Just dismiss it and move on. Now let's head back to our lonely tab that we left alone a while back. Hooray, we got traffic! At this point, you can do whatever you want with it. Whenever a network request is done from your phone it will appear in there. In the case for SLOWLY, it caches image files, so you may need to clear the app's cache/storage in order to see the stamp requests. Here's a ProTip: if you don't like how cluttered it is, you can clear it by pressing "Z". It's not documented anywhere, it just works. This has endless possibilities. You could get your own token to make a desktop notification system (though that may be against their terms of service). One last thought before I end the blog - you may notice an interesting URL in your network log. https://api.getslowly.com/slowly Well, that's awfully vague. What is that for? Well, for starters, it's a JSON page. And that's as much as I will tell you. It's up to you to find out what that page does. Have a good day, and happy mining! Category:Blog posts